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ABSTRACT 

The author describes ways to test for relationships 
between student achievement and higher cognitive questions asked by 
the teacher. He details methods for evaluating student responses in 
terms of answer quality and presents some of his own preliminary 
research on student response length. Higher cognitive questions, he 
bypothesizes, elicit and shape thought processes which manifest 
themselves in oral responses. Three tables of data are presented. 
(JB) 
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WHAT EFFECT DO TEACHERS' QUESTIONS HAVE ON STUDENTS? 



Heredlth D. Gall 
Far West Laboratory for Educational Research and Development 



In a literature review of questioning research a few years ago (Sail, 1970), 
I concluded that the time was ripe to focus our research efforts on investi- 
gating the effects which teachers' questioning practices have on student 
behavior. The majority of previous research has concentrated on descriptive 
analysis of teachers' questions or evaluation of training programs designed 
to change question-asking behavior* Much useful knowledge has resulted 
from these studies, but the value of this knowledge depends ultimately upon 
our ability to demonstrate that teacher questions help students to learn. 
What is the point of training teachers in techniques of questioning unless 
these techniques bring about demonstrable student learning? 

Barak Rosenshine's review of correlational research on teacher effects (1971) 
revealed few statistically significant or consistent relationships between 
student achievement gains and teacher use of higher cognitive questionsl 
or probing questions. 2 However, in the studies with which I am familiar, 
an inappropriate paper-and-pencil test was used to assess student learning. 
For example, in Wright and Nuthall's study (1970) a multiple-choice test 
was used to assess students' mastery of facts after having been taught 
a curriculum unit. Not surprisingly, the teachers' percentage of higher 
cognitive questions showed a slight negative correlation with this criterion. 
If a paper-and-pencil , multiple-choice test is an inappropriate measure for 
assessing the effects of higher cognitive questions, what would constitute 
a suitable measure? To my knowledge, there are no suitable measures 
currently available. In the following sections I will sketch the types of 
measures which I believe need to be developed in order to validly assess 
effects of higher cognitive and probing questions on student learning. 



Higher cognitive questions require students to think, for example, to 
analyze situations, solve problems, state opinions, or make predictions. 
They can be contrasted with fact questions, which require only recall 
of information or simple observation. 

Probing questions are designed to improve a student's initial response 
to another question. 



Within-Discussion Effects 

Me can start our search for effects of higher cognitive and probing 
questions within the discussion process Itself. Suppose that a teacher 
asks the higher cognitive -question, "Why do you think ecology has become 
so popular recently?" If the question has its presumed effect, it will 
cause the student to think of reasons to explain this phenomenon. However, 
since thought processes cannot be observed directly, we need to look for 
indirect effects in the characteristics of his verbal response. 

What characteristics should we assess? Unlike fact questions, I do not 
believe that higher cognitive questions have single correct answers, so they 
cannot be scored simply as correct or Incorrect. However, the answer could 
be scored for content . In the example above, we could score the student's 
answer for presence or absence of one or more "reasons" to explain the 
current popularity of ecology. I believe it is also desirable to score 
the answer for quality . Was it a good, satisfactory or poor answer? 

In the past a few studies have indirectly assessed the quality of a student 
answer by measuring its length (number of words). The assumption is that 
longer answers reflect more thought and verbal fluency than short answers. 
To my knowledge this assumption has not been empirically tested. However, 
I do have data which tests a related hypothesis: if higher cognitive 
questions indeed elicit thought whereas fact questions do not, and if 
length of an answer reflects the presence of thought, student answers to 
higher cognitive questions should be longer than their answers to fact 
questions. The data presented in Table 1 (from Gall, Dunning, Galassi, 
and Banks, 1970, pp. 41-42) substantiate this hypothesis. 

Although length of a student response may be a rough index of quality, 
more direct measures are needed. The following list of seven criteria 
may comprise some of the components which determine quality of response: 

1. CLARITY The student answers in understandable 

English without mumbling, failing to finish, 
or confusing his thoughts. 

2. ACCURACY The student's answer contains no factual 

errors and is based on accurate informa- 
tion. 



3. APPROPRIATENESS 

4. SPECIFICITY 

5. SUPPORT 



The student answers the question that was 
asked. 

The student clearly identifies who and what 
he is talking about. 

The student gives reasons, facts, or 
examples to support his statement, or he 
explains the criteria or assumptions on which 
he bases his opinion. 



TABLE 1 



LENGTH OF STUDENT RESPONSE TO FACT AND 
HIGHER COGNITIVE QUESTIONS 





MEAN NUMBER OF WORDS^ 


TREATMENT GROUP 


FACT 

QUESTIONS 


HIGHER COGNI- 
TIVE QUESTIONS 


4-6th Grade Teachers Before Training 
Experimental Group 1 
Experimental Group 2 
Control Group 


9.5 
7.4 
7.8 


15.0 
11.3 
11.5 


4-6th Grade Teachers After Training 
Experimental Group 1 
Experim-..nta1 Group 2 
Control Group 


7.8 
8.0 
8.7 


15.6 
13.2 
13.2 


7th Grade Teachers Before Training 
Experimental Group 1 
Experimental Group 2 
Control Group 


9.1 
11.0 
7.0 


16.0 
19.4 
11.9 


7th Grade Teachers After Training 
Experimental Group 1 
Experimental Group 2 
Control Group 


12.7 
11.3 
9.2 


18.9 
17.8 
13.2 



The Individual student response was the unit used In computing the 
mean. The range of N's for the means presented here Is 137-744. 



The student's answer shows that he is 
aware that there are many ways of looking 
at the problem being discussed, and that 
they must be considered before a val id 
judgment can be reached. 

The student draws upon current knowledge 
and past experience to create or discover 
ideas that are new to him. 

(Gall, Dunning, and Weathersby, 1971) 

To my knowledge, these criteria have not yet been validated to determine 
their usefulness in rating quality of student answers. 

Probing questions, properly used, should improve the quality of a student's 
initial response. Suppose that in response to the question, "Why do you 
think ecology has become so popular recently?", a student answers, "Because 
people are becomiiiy more aware of how they're mistreating nature." The 
teacher could probe this response to improve its quality by asking questions 
such as, "What people are you referring to?", "What do you mean by 'mis- 
treating'?", or "What are some examples?" Probably the same quality measure 
used to assess the effect of higher cognitive questions could be used to 
assess the effect of probing questions. Also, as with higher cognitive 
questions, length of response might be used as a rough index of quality 
of response to probing questions. In a recent study of preservice teachers 
(Saunders, Gall, and Smith, 1973), we tested the hypothesis that probing 
questions, properly used, will take an initially weak response and improve 
its quality, as measured by length. The data presented in Table 2 provide 
strong support for this hypothesis. 



6. COMPLEXITY 



7. ORIGINALITY 



Post-Discussion Effects 

If a student has learned from a discussion, it should be manifest in his 
behavior after the discussion has ended. If the student listened to his 
peers, he should have acquired a number of ideas not his own about 
questions raised by the teacher. Thus, one effect of a teacher's higher 
cognitive question, particularly when redirected to several students, 
might be to increase the student's repertoire of ideas pertaining to that 
question. Another effect might be that the student is able to give a 
higher quality answer to a given question after the discussion, if that 
question was covered in the discussion. The rationale for this hypothesized 
effect is that a good discussion provides the student with the opportunity 
to a) respond to the question, b) have his response challenged by a probing 
question, and/or c) listen to another student's response. A third effect 
of the discussion might be to change or modify the specific content of 
the student's response to a question; that is, he might hold to a different 



TABLE 2 



LENGTH OF STUDENT RESPONSE PRIOR TO AND 
FOLLOWING PROBING QUESTIONS 



> 

TREATMENT GROUP 


1 

NUMBER 
OF 

TEACHERS 


NUMBER 
OF 

PROBES 


MEAN NUMBER OF WORDS 


PRE-PROBE 


POST-PROBE 


(Before Training)^ 










Group 1 


13 


11 


4.6 


13.7 


Group 2 


14 


9 


7.1 


16.9 


Group 3 


15 


16 


13.1 


15.1 


Group 4 


13 


13 


6.7 


14.2 


Group 5 


12 


4 


10.5 


15.5 


Group 6 


16 


10 


4.6 


11.7 


(After Training) 










Group 1 


13 


39 


10.3 


16.0 


Group 2 


14 


26 


7.7 


15.5 


Group 3 


15 


27 


6.0 


18.6 


Group 4 


13 


25 


13.7 


15.9 


Group 5 


12 


13 


8.2 


12.8 


Group 6 


16 


19 


6.8 


9.4 



Training involved different methods of instruction in use of higher 
cognitive and probing questions. 
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prediction, reason, opinion, generalization, or solution after the 
discussion than before. 

How would we measure these effects? First, It would be necessary to test 
each student by himself or herself, since learning occurs In the Individual^ 
not In the group. The test could consist of eliciting the student's response 
to the same set of questions before and after the discussion; of course, 
this set of questions should be derived from those asked by the teacher 
during the discussion. Since the teacher's use of questioning techniques 
presumably shapes and Improves students' oral responses, the criterion test 
should also elicit oral responses .1 To determine whether Improvement In 
quality of oral discourse transfers to the written mode, the student could 
be tested by having him write essay*type answers to higher-cognitive questions 
asked by the teacher in the discussion. 



Long-Term Effects 

Thus far I have been concerned with Immediate effects of a teacher's use 
of higher cognitive and probing questions In a particular discussion. If 
a teacher makes habitual use of these techniques, what would the effect be 
on students? Suppose that a teacher uses these techniques in a discussion 
situation twice a week for four months. If a discussion In the first week 
is compared with a discussion in the last week of this time period, I pre- 
dict that the quality of student responses and amount of learning would be 
greater in the later discussion. 



An Experiment 

An experiment to test the various effects which I have described above Is 
represented schematically In Table 3. Experimental teachers are persons 
who naturally or by training make frequent use of higher cognitive and 
probing questions. Control teachers are persons who make Infrequent or no 
use of these techniques. The data collected using this experimental design 
could be analyzed to test the following hypotheses: 

a. Experimental teachers will elicit more high-quality student 
responses than control teachers. Compare X2 with Yo* 



In previous research teacher use of oral questioning techniques has been 
correlated with students' written performance. This design flaw probably 
accounts In part for some of the negative results that have been reported. 



ERIC 



TABLE 3 

EXPERIMENTAL DESIGN TO TEST FOR CHANGE IN 
QUALITY OF STUDENT RESPONSE 



TEST OCCASION 


EXPERIMENTAL 
GROUP 


CONTROL 
GROUP 


Immediate Pretest 


h 


\ 


Discussion 


h 




Imnedlate Posttest 


h 


h 


Interval of time during which experimental 
teachers make habitual use of higher cognitive 
? H oroblng questions. 


Immediate Pretest 


h 


»4 


Discussion 


h 


»5 


Immediate Posttest 


h 


»6 
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b. Students of experimental teachers will learn more from a 
given discussion than students of control teachers. Compare 
experimental group gain from pretest to posttest (X3-X^) with 
control group gain (Y3-Y^). 

c. With repeated exposure to higher cognitive and probing 
questions, students will Improve their ability to respond In 
discussion situations (compare X5 with Xo) and will learn 
more from the discussion (compare X5 with X3). Students of 
control teachers will not gain (compare Ye with Y«, and 

Ye with Y3). 



Summary 

My review of research on higher cognitive and probing questions Indicates 
that we know very little about the effect of these techniques on student 
learning. The main effect which I have hypothesized here Is that they 
elicit and shape thought procesi^es, which become manifest In the quality 
of the student's oral answer. Before research can be done to test this 
hypothesis, we need to define "quality" and to develop a reliable, valid 
measure for rating It, 



Rf^ERENCES 



Gall, Meredith D. The use of questions in teaching. Review of Educational 
Research . 1970. 40 . 707-721. 

Gall. Meredith 0.. Dunning. Barbara. Galassl. John, and Banks. Henry. 
Main field test report - Mini course 9 . San Francisco: Far West 
Laboratory for Educational Research and Development. 1970. 

Gall. Meredith D.. Dunning. Barbara, and Weathersby. Rita. Teachers hand- 
book; Higher cognitive questioning . New York: Macmlllan Educational 
Services, 1971. 

Rosenshine. Barak. Teaching behaviours and student achievement . London: 
National Foundation for E6.:atlonal Research in England and Wales, 
1971. 

Saunders, Walter, Gall, Meredith, and Smith, Gerrie. Preservice study of 
Minicourse 9. San Francisco: Far West Laboratory for Educational 
Research and Development, 1973 (report in progress). 

Wright, C.J. and Nuthall, G. Relationi'.hip between teacher behaviors and 
pupil achievement in three elementary science lessons. American 
Educational Research Journal , 1970, 7, 477-493. 



